Papers with bilingual lexicon induction
Joint Training for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing methods for learning cross-lingual word embeddings incorporate sub-word information during training. |
| Approach: | They propose a method that incorporates sub-word information during training to learn cross-lingual word embeddings from monolingual data and a bilingual lexicon. |
| Outcome: | The proposed method improves on bilingual lexicon induction, monolingual word similarity, and document classification using low-resource languages. |
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised neural machine translation (UNMT) use cross-lingual pretraining to align the lexical- and high-level representations of two languages. |
| Approach: | They propose to use type-level cross-lingual subword embeddings to enhance the bilingual masked language model pretraining with lexical-level information to align the two languages. |
| Outcome: | Empirical results show that the method improves on UNMT (up to 4.5 BLEU) and bilingual lexicon induction compared to baseline models. |
Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces (P19-1)
Copied to clipboard
| Challenge: | Recent work on bilingual lexicon induction (BLI) relies on an assumption about the isometry of two embedding spaces. |
| Approach: | They propose a semi-supervised approach that relaxes the isometric assumption while leveraging limited aligned bilingual lexicons and a larger set of unaligned word embeddings. |
| Outcome: | The proposed method obtains state-of-the-art results on 15 of 18 language pairs on the MUSE dataset and does particularly well when the embedding spaces don’t appear isometric. |
Evaluating a Joint Training Approach for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora on Lower-resource Languages (2021.starsem-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings provide a way for information to be transferred between languages. |
| Approach: | They propose a joint training approach that incorporates sub-word information during training to learn cross-lingual embeddings. |
| Outcome: | The proposed method improves bilingual lexicon induction, especially for out-of-vocabulary words (OOVs) it is able to represent out- of-vocal words (OVs) and is more isomorphic than previous methods. |
Evaluating bilingual word embeddings on the long tail (N18-2)
Copied to clipboard
| Challenge: | Bilingual word embeddings are useful for bilingual lexicon induction, but they focus on frequent words in general domains. |
| Approach: | They propose to evaluate bilingual word embeddings on rare words in different domains . they propose to use a multilingual dataset to build and combine BWEs based on a single word . |
| Outcome: | The proposed evaluations show that state-of-the-art methods fail on rare words . the proposed evaluation is based on a gold standard dataset and code . |
Anchor-based Bilingual Word Embeddings for Low-Resource Languages (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to build monolingual word embeddings rely on a cheap bilingual signal and monolingual data. |
| Approach: | They propose a method where the vector space of the high resource source language is used as a starting point for training an embedding space for the low resource target language. |
| Outcome: | The proposed approach improves bilingual lexicon induction performance and target language MWE quality. |
A Discriminative Latent-Variable Model for Bilingual Lexicon Induction (D18-1)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction take advantage of word embeddings, but our model is not as efficient as previous work. |
| Approach: | They propose a discriminative latent-variable model for bilingual lexicon induction that combines the bipartite matching dictionary prior and an embedding-based approach. |
| Outcome: | The proposed model outperforms existing models on six language pairs and shows that it mitigates hubness problem. |
Orthographic Features for Bilingual Lexicon Induction (P18-2)
Copied to clipboard
| Challenge: | Recent embedding-based methods do not take advantage of orthographic features, such as edit distance, which can be helpful for pairs of related languages. |
| Approach: | They propose to use orthographic features to integrate orthographic induction into embedding methods . they use document-aligned data instead of a seed dictionary to learn bilingual embedds . |
| Outcome: | This work extends embedding-based methods to incorporate orthographic features . it shows that the methods can learn bilingual embeddables in low-resource languages . |
An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces (2021.findings-emnlp)
Copied to clipboard
Kelly Marchisio, Youngser Park, Ali Saad-Eldin, Anton Alyakin, Kevin Duh, Carey Priebe, Philipp Koehn
| Challenge: | Existing work in bilingual lexicon induction views word embeddings as vectors in Euclidean space. |
| Approach: | They propose to use word embeddings as nodes in a weighted graph to examine a node’s graph neighborhood without assuming a linear transform. |
| Outcome: | The proposed approaches are compared under different data conditions and show that they complement each other when combined. |
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLEs) are used for downstream NLP tasks . CLEs are based on bilingual lexicon induction (BLI) evaluations vary greatly, hindering ability to interpret performance and properties of different CLE models. |
| Approach: | They evaluate CLE models for a large number of language pairs on bilingual lexicon induction and three downstream tasks. |
| Outcome: | The proposed model performance is based on supervised and unsupervised models on bilingual lexicon induction and three downstream tasks. |
Two Methods for Domain Adaptation of Bilingual Tasks: Delightfully Simple and Broadly Applicable (P18-1)
Copied to clipboard
| Challenge: | Previously, domain adaptation approaches to bilingual tasks were proposed . we show that simple adaptation process involving only unlabeled text is highly effective . |
| Approach: | They propose a method for domain adaptation of bilingual word embeddings using unlabeled data . they then tailor a semi-supervised classification method from computer vision to these tasks . |
| Outcome: | The proposed method improves on two bilingual tasks using unlabeled data. |
Revisiting the Context Window for Cross-lingual Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to mapping-based cross-lingual word embeddings are based on the assumption that the source and target embeddable spaces are structurally similar. |
| Approach: | They propose to use different context windows to evaluate bilingual word embeddings in various languages, domains, and tasks. |
| Outcome: | The size of both the source and target window improves bilingual lexicon induction, especially on frequent nouns. |
Improving Bilingual Lexicon Induction for Low Frequency Words (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate word-level translations from non-parallel corpora are based on word embeddings. |
| Approach: | They propose two methods to address two factors that degrade bilingual lexicon induction accuracy . they propose a method that assumes a seeding dictionary is available . |
| Outcome: | The proposed method improves bilingual lexicon induction significantly for rare words. |
Density Matching for Bilingual Word Embedding (N19-1)
Copied to clipboard
| Challenge: | Recent approaches to cross-lingual word embeddings have been based on linear transformations between the embeddable vectors in the two languages. |
| Approach: | They propose a method that expresses two monolingual embedding spaces as probability densities and matches them using a Gaussian mixture model. |
| Outcome: | The proposed method can achieve competitive or superior performance on bilingual lexicon induction and cross-lingual word similarity data. |
Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on bilingual lexicon induction fails in low-resource scenarios . a language dataset is considered low- resource based on its own embedding space . |
| Approach: | They propose a graph-matching method that improves bilingual lexicon induction performance across 40 language pairs using optimal transport. |
| Outcome: | The proposed method is especially strong with low amounts of supervision. |
Evaluating Word Embeddings with Categorical Modularity (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing word embeddings use different bilingual supervision signals with varying levels of strength. |
| Approach: | They propose a graph modularity metric to measure word embedding quality . they use a set of 500 words belonging to 59 neurobiologically motivated semantic categories . |
| Outcome: | The proposed metric measures word embedding quality on monolingual and cross-lingual tasks. |
The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have suggested that bilingual lexicon induction is influenced by the (dis)similarity of the languages at hand. |
| Approach: | They propose to measure the isomorphism of monolingual embedding spaces based on their spectra and introduce isometric measures to measure their similarity. |
| Outcome: | The proposed measures outperform standard isomorphism measures while being more tractable and easier to interpret. |
Learning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs (N19-1)
Copied to clipboard
| Challenge: | Recent research has found that a shared bilingual word embedding space can be induced by projecting monolingual word embeds from two languages without any bilingual supervision. |
| Approach: | They propose a framework for learning unsupervised multilingual word embeddings that mitigates instability issues for distant language pairs. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on two downstream tasks outperforming even supervised baselines. |
Why Overfitting Isn’t Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies only evaluate cross-lingual word embeddings on bilingual lexicon induction (BLI) however, underfitting can hinder generalization to other downstream tasks. |
| Approach: | They retrofit cross-lingual word embeddings to the training dictionary and a synthetic dictionary to improve their results. |
| Outcome: | The proposed method improves accuracy on two downstream tasks, despite underfitting the training dictionary. |
Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for building high-quality CLWEs learn mappings that minimise the l2 norm loss function but this optimisation objective has been shown to be sensitive to outliers. |
| Approach: | They propose a simple post-processing step to improve cross-lingual word embeddings using the Manhattan norm goodness-of-fit criterion. |
| Outcome: | The proposed approach outperforms four state-of-the-art baselines in bilingual lexicon induction and cross-lingual transfer tasks. |
A Simple Approach to Learning Unsupervised Multilingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised cross-lingual embeddings in the bilingual setting has given the impetus to learning a shared embeddable space for several languages. |
| Approach: | They propose to solve two sub-problems together to learn a shared embedding space for several languages. |
| Outcome: | The proposed approach outperforms existing methods in bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing tasks. |
Morphologically Aware Word-Level Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Current approaches to bilingual lexicon induction (BLI) ignore inflectional morphology . current models degrade when translating less frequent inflected forms . |
| Approach: | They propose a morphologically aware probability model that models lexeme translation and inflectional morphology in a structured way. |
| Outcome: | The proposed model improves performance across 6 language pairs and in the weakly supervised setting. |
Unsupervised Cross-lingual Transfer of Word Embedding Spaces (D18-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word mapping require cross-linguistic supervision, but this is not available for many low resource languages. |
| Approach: | They propose an unsupervised method that learns transformation functions over corresponding word embedding spaces using a distributed distributional matching algorithm. |
| Outcome: | The proposed method performs better on bilingual lexicon induction and cross-lingual word similarity prediction datasets than other supervised and unsupervised methods. |
How Lexical is Bilingual Lexicon Induction? (2024.findings-naacl)
Copied to clipboard
| Challenge: | lexical variation and low-resource settings make it difficult to learn in low-level settings. |
| Approach: | They propose to incorporate additional lexical information into the retrieve-and-rank approach to improve lexicon induction. |
| Outcome: | The proposed approach improves on XLING by an average of 2% across all language pairs. |
A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon Induction (2020.acl-main)
Copied to clipboard
| Challenge: | Recent methods for bilingual lexicon induction are based on unsupervised cross-lingual word embeddings . previous methods only use word-level information, which is limited and inaccurate . |
| Approach: | They propose a graph-based approach to induce bilingual lexicons in a coarse-to-fine way . they use word cliques from graphs and aligned clique-level words to find initial translation solution . |
| Outcome: | The proposed method improves the performance of bilingual lexicon induction compared with previous methods. |
Evaluating Sub-word Embeddings in Cross-lingual Models (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to learning sub-word embeddings for out-of-vocabulary words have not considered sub- word embedds in cross-lingual models. |
| Approach: | They propose to use sub-word embeddings to form cross-lingual embeddables for out-of-vocabulary (OOV) words for which no embeddibles are available. |
| Outcome: | The proposed bilingual lexicon induction task shows that sub-word embeddings can be leveraged to form cross-lingual embeddables for OOV words. |
Prix-LM: Pretraining for Multilingual Knowledge Base Construction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to build and enrich multilingual knowledge bases have not been successful . knowledge expressed in different languages may be complementary and unequally distributed . |
| Approach: | They propose a model that integrates useful multilingual and KB-based factual knowledge into a single model. |
| Outcome: | The proposed model can provide richer combined knowledge than monolingual KBs. |
IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces (2022.emnlp-main)
Copied to clipboard
| Challenge: | a faulty cross-lingual mapping technique results in non-isomorphic word embedding spaces . we address the root-cause of this problem by increasing the relative isomorphism of word embedsing spaces. |
| Approach: | They address the root-cause of faulty cross-lingual mapping by incorporating global measures of isomorphism into the skipgram loss function. |
| Outcome: | The proposed method improves bilingual lexicon induction under domain mismatch and with training algorithm dissimilarities. |
Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to extract named entity datasets from parallel corpora require large monolingual corporata or word aligners that are unavailable or perform poorly for underresourced languages. |
| Approach: | They propose a method for creating a multilingual named entity resource from parallel corpora and apply it to the Parallel Bible Corpus, a corpus of more than 1000 languages. |
| Outcome: | The proposed method outperforms existing methods in two tasks. |
Massively Multilingual Lexical Specialization of Multilingual Transformers (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work focused on lexical specialization of monolingual PLMs with immense quantities of monolinguistic constraints, but recent work shows that pretrained language models can be rewired to produce high-quality word representations and perform type-level lexicals. |
| Approach: | They propose to expose massively multilingual transformers to multilingual lexical knowledge at scale using BabelNet as a source of multilingual and cross-lingual type-level lexicon knowledge. |
| Outcome: | The proposed method shows that pretrained language models can be rewired to produce high-quality word representations and perform type-level lexical tasks. |
Do We Really Need Fully Unsupervised Cross-Lingual Embeddings? (D19-1)
Copied to clipboard
| Challenge: | a series of bilingual lexicon induction experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods fail for a large number of language pairs. |
| Approach: | They propose to use fully unsupervised approaches to project monolingual embeddings into a shared cross-lingual space without any cross-linguistic signal. |
| Outcome: | The proposed methods fail for a large number of language pairs, but never surpass weakly supervised methods. |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |
Bilingual Lexicon Induction through Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction use nearest neighbor or related retrieval methods to induce word translation pairs. |
| Approach: | They propose a method that aligns word embeddings in two languages and uses them to build a phrase-table and a language model to extract the bilingual lexicon. |
| Outcome: | The proposed method improves accuracy 6 points over nearest neighbor and 4 points over CSLS retrieval on the same cross-lingual embeddings. |
Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context Anchoring (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embedders. |
| Approach: | They propose an unsupervised mapping approach that fixes fixed embeddings and learns new ones for the source language that are aligned with them. |
| Outcome: | The proposed method outperforms conventional mapping methods on bilingual lexicon induction and obtains competitive results in the downstream XNLI task. |
Data Selection for Bilingual Lexicon Induction from Specialized Comparable Corpora (2020.coling-main)
Copied to clipboard
| Challenge: | Narrow specialized comparable corpora are small in size, making it difficult to build efficient models to acquire translation equivalents. |
| Approach: | They propose to use Tf-Idf and cross entropy to improve bilingual lexicon induction from specialized comparable corpora by a factor of 10 . |
| Outcome: | The proposed methods improve bilingual lexicon induction by a large margin. |
Classification-Based Self-Learning for Weakly Supervised Bilingual Lexicon Induction (2020.acl-main)
Copied to clipboard
| Challenge: | Effective projection-based cross-lingual word embedding induction relies on the iterative self-learning procedure. |
| Approach: | They propose a classification-based approach to self-learning that allows for integration of diverse features into the iterative process. |
| Outcome: | The proposed method improves bilingual lexicon induction on a weakly supervised setup with 28 language pairs. |
Non-Linear Instance-Based Cross-Lingual Mapping for Non-Isomorphic Embedding Spaces (2020.acl-main)
Copied to clipboard
| Challenge: | InstaMap is a non-parametric model that learns a global linear projection by iterating between finding an optimal rotation of the source embedding space and moving each point along an instance-specific translation vector. |
| Approach: | They propose a non-parametric model that learns a global linear projection by iteratively finding an optimal rotation of the source embedding space and moving each point along an instance-specific translation vector. |
| Outcome: | The proposed model performs better than previous models on bilingual lexicon induction across a set of 28 diverse language pairs. |